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The Honorable Charles E. Schumer 
United States Senate 
313 Hart Senate Office Building 
Washington, DC 20510 

Dear Senator Schumer: 

On June 11, 2003, the Office of Inspector General (OIG) for the Department of 
Homeland Security (DHS) received a copy of a letter, dated May 9, 2003, sent to you 
by Admiral James M. Loy, Administrator of the Transportation Security 
Administration (TSA). 1 Admiral Loy's letter presented the results of TSA's Office of 
Internal Affairs and Program Review (OIAPR) investigation into media allegations 
that checked baggage security screeners being trained to operate explosive detection 
system machines, at La Guardia Airport in New York, were given all or most of the 
answers to their certification tests before the tests were administered. The OIAPR 
reported that its interviews and investigation established that 22 of the 25 questions 
appearing in the final exam were also used for the daily lesson quizzes. OIAPR 
further reported that the testing and training was conducted as prescribed by TSA and 
that it found no misconduct on the part of the instructors. 

Upon learning of the OIAPR review and TSA's letter of May 9, 2003, 1 wrote to you to 
advise that my staff would review the work performed by TSA and advise you of the results 
of our review as soon as it is completed. I took this action because it seemed incongruous to 
me that TSA would confirm this pattern of testing but limit its consideration of the 
allegations to a carefully circumscribed conclusion that no misconduct occurred. 

In the ensuing weeks, my staff reviewed TSA's internal investigation and other work 
conducted regarding this matter. We met with TSA's OIAPR Director as well as the 
Program Analyst and Lead Inspector involved in the internal review. We reviewed and 
analyzed the supporting documentation collected by OIAPR for its internal report, including 
contracts, work papers, interviews, affidavits, training curriculum, quizzes and examinations, 
and applicable operating procedures. In addition, we reviewed the investigation conducted 
by OIAPR for the purpose of confirming or commenting on the "competency of the OIAPR 
inquiry." During the course of our work, we received complete cooperation from OIAPR. 



1 TSA's letter of May 9, 2003, indicated that the OIG for DHS was to receive a copy, but we did not. We 
received a copy when the DOT/OIG provided a copy to us during a meeting on other subjects on June 11, 2003. 



Results of the OIG Review 



First, we concluded that the investigation performed by OIAPR was conducted in a thorough 
and professional manner. The interviews and affidavits taken from witnesses and summaries 
of documents pertinent to the inquiry were reported accurately in the OIAPR Report of 
Investigation and supported the conclusion that no misconduct by the certified instructors 
was disclosed. 

Second, we concluded that TSA failed to recognize or acknowledge the extremely significant 
questions raised by its chosen method of training and testing. 

As OIAPR reported, TSA contracted with Boeing Services Company to train approximately 
30,000 checked baggage screeners to operate explosive detection systems (EDS) and 
explosive trace detection (ETS) equipment. The contract further provided that TSA was 
responsible for supplying the training and testing materials Boeing would use for the training. 
TSA contracted with Northrop- Grumman to develop the curriculum and tests. Under the 
latter contract, Northrop- Grumman developed a 40- hour curriculum consisting of 5 modules. 
Each module was followed by an open-book quiz of 5 questions. At the conclusion of the 
course, a closed-book final examination was given that consisted of 25 questions. The 
OIAPR investigation reported that 22 of the 25 final examination questions were identical to 
the questions used for the quizzes and that the quiz questions were "often" part of the final 
review that preceded administration of the final examination. 

We also reviewed the questions used in TSA's training of baggage screeners. We concluded 
that 19 of the 25 questions for the EDS 3DX 6000 machine quiz were identical or virtually 
identical to the questions on the final examination. 2 Of the six remaining questions, three 
questions were similar and three questions were not repeated on the final test. 

We also evaluated the quiz and examination questions for the EDS CTX 5500/2500 and EDS 
CTX 9000 machines. We found 13 of the 25 questions were identical in each final 
examination. According to the instructor interviews conducted by OIAPR, the instructors 
were not provided with advance copies of the final examination, but they were provided with 
TSA-prescribed lesson review questions and answers. The instructors stated that they had 
followed the TSA course outline and curriculum in preparing the students for the final test. 

TSA instituted a training and testing method that prepared each student to answer a specific 
set of questions. The protocol that TSA adopted maximized the likelihood that students 
would pass. TSA's approach maximized its capacity to push students through school and 
certify them as screeners of checked baggage for assignment to airports. By prepping the 
students for the final examination questions, however, TSA lost the opportunity to fully 
evaluate the student's mastery and comprehension of the course material. 



2 Included in this category was one question that was stated in the affirmative with a correct answer of "true" 
that was recast in the negative with a correct answer of "false". 



We did not evaluate the entire curriculum and course material provided to prospective 
baggage screeners. However, our review of the content and phrasing of the examination 
questions points to a second concern that also undermined the soundness of the testing. 
Some questions give away the answer and others are simply inane. To illustrate, the multiple 
choice questions below are taken from the final examination for the EDS CTX 5500/2500 
course. 



Question: "What is the role of a detonator in an explosion?" Correct answer: "Creates a 
small explosion that detonates the main explosive charge." (Emphasis added). 

Question: "What type of weapons are explosives detection systems, such as the TX 
5500/2500, designed to detect?" Correct answer: "Improvised Explosive Devices (IEDs)." 
(Emphasis added). 

Question: "Clocks, watches, and integrated circuit chips are examples of which IED 
component?" Correct answer: "Switch/ timer." (Emphasis added). 



Question: "Why is it important to screen bags for IEDs?" 

a. The IED batteries could leak and damage other passenger bags. 

b. The wires in the IED could cause a short to the aircraft wires. 

c. IEDs can cause loss of lives, property, and aircraft. 

d. The ticking timer could worry other passengers. 



Question: How do threats get aboard an aircraft? 

a. In carry-on bags. 

b . In checked- in bags. 

c. In another person's bag. 

d. All of the above. 



Not all the questions are of this type. Many questions asked for information relative to the 
operation of the machine, such as how to respond to fault or shutdown events and logout 
procedures, or called on the student to know the components of a typical explosive device, 
i.e., main charge, power source, timer, and detonator. Notwithstanding that some questions 
addressed information material to the course, it is extremely disturbing that most of the 
questions were rehearsed before the final examination, that a number of the questions were 
phrased so as to provide an obvious clue to the correct answer, and other questions appear to 
be simplistic. 

Finally, there is another, critical defect in the TSA testing and certification process. Of the 
examination materials provided to us by TSA, not a single question called upon a student to 
demonstrate a sufficient mastery of the class content to achieve the purpose of the training, 
i.e., to examine a real or simulated EDS screen to identify correctly objects in baggage that 
constitute threats or are prohibited. 

On June 12, 2003, TSA's Office of Training and Quality Performance issued an internal 
memorandum that provided information on steps taken by TSA to update, modify, and 



improve the training of checked baggage screeners. These actions include: (1) a complete 
review of the program and the refinement of areas as needing improvement; (2) a revamping 
of all quizzes and formal test material to ensure there is no exact duplication of questions; 
and (3) a review of the updated testing materials to ensure test validity. These actions may 
strengthen TSA's a training program for checked baggage screeners. We will conduct a 
follow-up review of TSA's training and testing methods to evaluate the effectiveness of the 
changes TSA announced it would make. 

In addition to our prospective review of TSA's training program for checked baggage 
screeners, we currently are auditing TSA's activities with respect to the maintenance and 
calibration of the explosives detection equipment. In addition, we are auditing the Boeing 
contract that guided TSA in its installation of this equipment at airports. Further, we are 
preparing an inspection report on the background investigations performed on passenger and 
baggage screeners and an audit report on TSA's management of the contracts used to acquire 
recruitment and hiring services. We will provide you the results of this work when it is 
completed. 

In the interim, should you wish to discuss our work regarding TSA's training program for 
checked baggage screeners, please contact me at 202-254-4100. 



Clark Kent Ervin 
Acting Inspector General 

cc: Admiral James M. Loy, Administrator of the Transportation Security Administration 



Sincerely, 





